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© Broadcast program Identification method and 

® A method and apparatus for identifying repet- 
itively broadcast programs, such as commercial ad- 
vertisements broadcast on television and radio, de- 
tects the occurrence of predetermined events in a 
video signal and/or in an audio signal, detects a 
stability condition and extracts a signature from the 
video and/or audio signal. The signatures and the 
times of occurrence of the signatures are stored and 
subsequently compared with reference signatures to 
identify the program and Its time of broadcast The 
reference signatures may be obtained by identifying 
repetitively extracted, unidentified signatures and in- 

gdexing a recording of the video and/or audio signals 
to enable the system user to identify the unidentified 
broadcast program and store the previously unidenti- 
Qfied signatures as additional reference signatures in 
C0 memory. 
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BROADCAST PROGRAM IDENTIFICATION METHOD AND APPARATUS 



BACKGROUND OF THE INVENTION 

1. Held of the Invention 

This invention relates generally to methods and 
apparatus for identifying repetitively broadcast pro- 
grams, and more particularly to a method and 
appaatus for identifying repetitively broadcast pro- 
grams, such as commercial advertisements broad- 
cast on television and radio. 



2. Description of the Prior Art 

Systems for identifying programs such as tele- 
vision and radio programs that are broadcast are 
known. Such systems fall Into various categories. 
These categories include manual systems wherein 
a clerk keeps a log indicating the programs broad- 
cast by a given station; systems that detect iden- 
tifying signals present in the television or radio 
broadcast signal; and systems that monitor and 
analyze the program content of the broadcast sig- 
nal utilizing pattern recognition and correlation 
techniques to identify the program. While all of 
these systems provide a way. to identify programs 
and to confirm that commercial advertisements are 
made available for public viewing, the manual 
methods are slow in acquiring data and are prone 
to inaccuracies resulting from the entry of erro- 
neous data that may be intentionally or unintention- 
ally entered. 

Systems that detect identification signals en- 
coded on the broadcast signal require the coopera- 
tion of the broadcasters who must encode the 
programs prior to broadcast for such systems to be 
effective. Consequently, such systems are useable 
only on a limited number of broadcasts, such as 
network broadcasts. Other known systems monitor 
and analyze the content of the program teeff, but 
heretofore the information processing requirements 
of such systems rendered such systems imprac- 
tical or uneconomical because of the computing 
capacity required to analyze the vast quantity of 
information present in broadcast signals. Moreover, 
previous attempts to reduce the quantity of in- 
formation processed tended to introduce inaccura- 
cies. 

An example of a system that utilizes the pro- 
gram content of a broadcast signal to identify the 
program Is described in United States Patent No. 
3,919,479 to Moon et aJ. The Moon et al. system 
utilizes a non-linear analog transform to produce a 
low-frequency envelope waveform, and the Infbr- 



mation in the low-frequency envelope of predeter- 
mined time interval is digitized to generate a signa- 
ture. The signatures thus generated are compared 
with reference signatures to identify the program, 
s However, the Moon et al. system generates the 
signatures continuously, and consequently, re- 
quires a large computer to process the data thus 
generated. 

Another system that monitors the program con- 

70 tent of a broadcast signal is disclosed in United 
States Patent No. 4,230,990 to Lert, Jr. et al. The 
system disclosed in Lert Jr. et al. patent reduces 
■ the amount of data that must be processed, as 
compared to the data amount to be processed by 

75 the Moon et al. patent, by utilizing cues that are 
either externally generated or present in the pro- 
gram material to initiate the signature generation 
and correlation process only after a cue has been 
detected. However, in the system disclosed in the 

20 Lert Jr. et al. patent a signature is extracted at a 
predetermined time following a detected cue sig- 
nal, and inaccuracies may be introduced if the 
signal is not stable at the time of signature extrac- 
tion or the extracted signature must include a large 

25 quantity of data for identification of the program 
with the desired accuracy. 



SUMMARY OF THE INVENTION 

30 

Based on the foregoing considerations the in- 
vention as claimed solves the problem of accu- 
rately identifying programs with a minimal amount 
of computation utilizing data reduction and pattern 

38 recognition techniques on the program material. 

Briefly, in accordance with a preferred embodi- 
ment of the Invention, there is provided a system 
having a station monitoring unit that monitors a 
particular broadcast source. The station monitoring 

40 unit includes a receiver for receiving programs 
broadcast on television or radio. The receiver is 
tuned to a particular broadcast source to be mon- 
itored. The monitored signals of the receiver are 
applied to audio processing circuitry, video pro- 

46 ceasing circuitry and a video recorder. The pro- 
cessed video signals and the processed audio sig- 
nals are applied to an events detector. Also, the 
processed audio signals are applied to audio signa- 
ture extraction circuitry and the processed video 

so signals are applied to video signature extraction 
circuitry. Both the audio and video signature ex- 
traction circuitry include an enable input coupled to 
the events detector. Upon the detection of one of a 
plurality of predetermined events, the events detec- 
tor thereafter detects a stability condition in the 
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monitored signal and then enables the signature 
extraction circuitry. The extracted signatures from 
both the audio signature extraction circuitry and the 
video signature extraction circuitry are combined 
according to a predetermined format to form a 
composite extracted signature that is stored. The 
time of occurrence of the thus extracted signature 
is stored and subsequently the extracted signature 
is compared with reference signatures to identify a 
broadcast program and Its time of broadcast The 
reference signatures are obtained by identifying 
repetitively extracted, unidentified signatures and 
assigning a temporary identification code to enable 
a system user to view the recorded program on a 
video recording and provide identification for the 
new reference signatures. 



DESCRIPTION OF THE DRAWING 

These and other objects and advantages of the 
present invention will become readily apparent 
upon consideration of the following detailed de- 
scription of a preferred embodiment of the present 
invention in connection with the attached drawing, 
wherein: 

FIG. 1 is a block diagram of a broadcast 
station monitoring unit of a broadcast program 
identification system according to the present in- 
vention; 

FIG. 2 is a block diagram of a recognition 
and post-processing unit of a broadcast program 
identification system according to the invention; 

RGS. 3 and 4 depict various waveforms 
associated with the block diagram of FIG. 1; 

FIG. 5 is a flow chart illustrating the logical 
steps performed by the station monitoring unit of 
FIG. 1; and 

RGS. 6 and 7 are flow charts illustrating the 
logical steps performed by the recognition and 
post-processing unit of RG. 2. 



DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

Referring now to the drawing, with particular 
attention to RG. 1, there is illustrated a block 
diagram of a station monitoring unit of the system 
according to the invention designated generally by 
the reference character 10. The station monitoring 
. unit 10 receives television broadcast signals from a 
receiving antenna 12. While the station monitoring 
unit 10 is depicted and generally described herein 
for monitoring a television broadcast signal which 
includes both video and audio signals, the princi- 
ples of the present invention disclosed herein are 
equally applicable to a broadcast radio signal. The 



received signals are applied to a receiver 14 which 
includes a tuner and amplifier. The receiver 14 is 
tuned to a desired broadcasting station to be mon- 
itored to provide a desired program signal 16 at the 

s output of receiver 14. The program signal 16 is 
applied to audio processing circuitry 18, video pro- 
cessing circuitry 20 and a recorder 22, such as a 
video cassette recorder or a VCR 22. The program 
signal 16 is recorded on a video tape 24 within the 

to VCR 22. 

The audio processing circuitry 18 includes an 
energy detector and a plurality of bandpass filters, 
and provides a detected energy signal 25 and, a 
plurality of signals 26 representative of an audio 

75 spectrum that may be, for example, in a range 
between 330 Hz and 2.54 kHz. An ASA^16 speech 
preprocessor integrated circuit manufactured by In- 
terstate Electronics Corporation of Orange, Califor- 
nia, may be employed to provide the bandpass 

20 filtering of audio processing circuitry 18. The audio 
spectrum signals 26, for example, 12 bandpass 
signals, are applied to a multiplexer 28 which pro- 
vides the analog spectrum signals to an analog-to- 
digital (A/D) converter 30. The A/D converter 30 

25 generates digital representations of the audio spec- 
trum signals or digitized audio signals 32. The 
digitized audio signals 32 are applied to both an 
audio signature extraction circuit 34 and to an 
events detector 38. The detected energy signal 25 

$o is applied to the events detector 38. The events 
detector 38 has an output that is coupled to an 
enable input 36 of the audio signature extraction 
circuitry 34. The audio signature extraction circuitry 
34 extracts an audio signature from the digitized 

35 audio signals 32 when enabled by the events de- 
tector 38 which is discussed subsequently in great- 
er detail. 

The video processing circuitry 20 includes a 
detector and a low-pass filter having, for example, 

40 a cut-off frequency of approximately 0 to 2 kHz, 
and provides a signal representative of the en- 
velope of the video signal to an analog-to-digrtal - 
(A/D) converter 40 which generates a digital repre- 
sentation of the video envelope. The digitized video 

45 envelope signal is applied to both a video signature 
extraction circuit 42 and the events detector 38 
which detects predetermined events that occur in 
the digitized video signal, and causes the signature 
of the digitized video signal to be extracted by 

so video signature extraction circuitry 42. The signa- 
ture thus generated is stored in a data storage 
device or memory 44 along with a time that the 
signature was extracted. The extraction times are 
provided by a clock 46 which provides a digital 

55 representation of time to the data storage device 
44. via, for example, a combining circuit 48. The 
clock 46 may be a real time clock or a relative time 
clock that is periodically reset from an accurate 
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clock, for example, a timing signal transmitted by 
the National Bureau of Standards. The dock 46 
provides the digital representation of time to a 
video date and time code generator 50 which ap- 
plies the digital representation of time to the VCR 
22 for storage on the video tape 24. The function of 
the events detector 38, the audio signature extrac- 
tion circuitry 34, the video signature extraction cir- 
cuitry 42, the data storage 44 and the clock 46 can 
be per formed by individual circuits as shown or by 
a microcomputer, a personal computer, or various 
microprocessor based systems. 

Referring to FIG. 2, there is shown a block 
diagram illustrating a signature recognition and 
post-processing unit designated generally by the 
reference character 54. The station monitor unit 10 
(RQ. 1), extracts and stores signatures and times 
of extraction thereof, and applies the signatures to 
the signature recognition and post-processing unit 
54 which compares the extracted signatures from 
the various station monitoring units 10 with a plural- 
ity of reference signatures taken from identified 
programs in order to identify particular broadcast 
programs corresponding to the extracted signa- 
tures received from the station monitoring units 10. 
The station monitoring units 10 may be interro- 
gated at periodic intervals by a data communica- 
tion circuit 56 that accesses the various station 
monitoring units 10 via a suitable communications 
system, for example an RS-232 Serial communica- 
tion fine, a common computer bus or a plurality of 
telephone lines 5a 

The extracted signatures thus collected are 
compared by a signature recognition computer 60 
to a plurality of reference signatures stored in a 
memory or reference signature data base 62. The 
signature recognition computer 60 performs the 
collection and classification of the signatures re- 
ceived from the station monitoring units 10, as weU 
as the generation of reference signatures to be 
stored in the reference signature data base 62. The 
signature recognition computer 60 compares the 
extracted signatures to the reference signatures 
stored in data base 62 and classifies and stores the 
signatures in a recognition file 64 which is coupled 
to a post-processing computer 66. The signature 
recognition computer 60 utilizes successive recog- 
nized extracted signatures to identify a particular 
program. The recognized extracted signatures from 
the various station monitoring units 10 are applied 
to the post-processing computer 66. The post- 
processing computer 66 produces a file or log 70 
of the Identified programs, commercial advertise- 
ments or other repetitively broadcast programs 
from each monitored station and can include the 
starting time of each identified or recognized pro- 
gram and the duration of the program. In addition, 
such analytical operations can provide various logi- 



cal checks to flag incomplete or otherwise defec- 
tive broadcasts of repetitively broadcast programs, 
and thereby enable the detection of possible false 
signal recognitions. . 
5 Occasionally, signatures that do not match any 
reference signatures will be extracted. Such signa- 
tures may be flagged by the system, particularly if 
such signatures occur between blank frames, and 
more particularly, If such blank frames occur at 30- 

io second intervals or less or multiples of 30-second 
intervals indicating that the unidentified signatures 
probably are part of a commercial message. The 
signatures are then compared against each other to 
find repetitive signatures and to avoid duplication. 

T5 The post-processing computer 66 assigns a tem- 
porary code to identify each repetitive non-recog- 
nized or unmatched extracted signature and stores 
the time of occurrence of the unidentified program. 
Thereby, a system user is enabled to view the 

ao video recording from the video tape 24 and Identify 
new broadcast programs. A data entry, terminal 68 
is employed to update the reference signature data 
base 62 to add new reference signatures so that 
subsequent broadcast of the repetitively broadcast 

26 program elements are correctly identified. 

Referring now to RQ. 3, there are shown 
waveforms illustrating the operation of the station 
monitoring unit 10 of FIG. 1. The output of the 
video processing circuitry 20 that is applied to the 

30 A/D converter 40 is shown as a video envelope 72. 
The A/D converter 40 takes a plurality of samples 
74, for example, 56 samples taken during a se- 
lected portion of a single frame interval, and gen- 
erates a digital value representative of the am- 

35 piltude of each of the samples 74. The digital 
values thus generated are applied to the events 
detector 38 (RQ. 1) and to the video signature 
extraction circuit 42 and are used to detect the 
certain predetermined events, to detect the stability 

40 condition following the ocairrence of the particular 
predetermined event and to extract signatures. 

The video signature extraction circuit 42 op- 
erates on the digitized samples 74 by taking the 
average value of consecutive samples 74, for ex- 

46 ample, two consecutive samples to generate a total 
of, for example, 28 average values or features 78 
that comprise an Image signature. Typically four 
bits define a feature, and consequently, an image 
or video signature can be defined by 112 bits of 

so data for the example discussed above; however, in 
some instances more features and more data may 
be required. 

Similarly, an audio spectrum. 78 is shown as 
the output of the audio processing circuit 18 that 

58 operates, for example, in the range between 330 Hz 
and 2£4 kHz to form 12 bandpass signals 80 
which are digitized by the A/D converter 30. The 
audio signature extraction circuit 34 operates on 
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the digitized bandpass signals 80 to generate a 
total of, for example, 12 features 82 that comprise 
an audio signature. By utilizing four bits to define a 
feature, the audio signature can be defined by 48 
bits. As described hereinbefore, the audio signa- 
tures are extracted from the amplitude of the 
digitized audio signals when enabled by the events 
detector 38. The audio signature extraction circuit 
34 typically is enabled a predetermined time after 
the video signature extrac tion circuit 42 is enabled, 
for example 1/3 second later. The video envelope 
signature corresponding to the video features 76 
and the audio spectrum signature corresponding to 
the audio features 82 advantageously can be com* 
bined into a composite signature. The extracted 
composite signature is formed, for example, by 
concatenating the video and audio signatures into a 
signature string of 160 bits. 

Referring now to FIG. 4, there are shown 
waveforms 84, 86 illustrating two examples of the 
processed broadcast signal that is applied to the 
events detector 38 via A/D converter 40. The 
events detector 38 compares the digitized samples 
74 of a current frame i with corresponding digitized 
samples 74 of a previous frame, either the imme- 
diately proceeding frame i-l or another previous 
frame i-k, where k, is used to indicate a variable 
length of time as may be determined, for example, 
by counting synchronization pulses. The detected 
audio energy signal 25 (FIG. 1) is applied to the 
events detector 38 for detecting specific events, for 
example, blank frames. 

Referring first to the broadcast signal desig- 
nated 84 in FIG. 4, blank frames are illustrated 
within the waveform 84. At least one blank frame is 
commonly used in the United States to separate 
commercial advertisements from other program 
material and during the blank frame there is gen- 
erally silent audio. The events detector 38 com- 
pares the digitized samples 74 (FIG. 3) and the 
detected audio energy signal 25 (FIG. 1) with pre- 
determined threshold values and thereby detects a 
blank frame wherein the sum of the digitized sam- 
ples 74 are less than the predetermined threshold 
values for a time period corresponding to a frame. 
After a blank frame is detected, the digitized sam- 
ples 74 of a subsequent frame are compared to a 
second threshold value to detect a non-blank frame 
before a signature is extracted from the broad cast 
signal. 

The events detector 38 detects other predeter- 
mined events, such as slow scene or gradual 
scene changes. A slow scene change occurs dur- 
ing a sequence of frames in which one image is 
gradually replaced by the superimposition of an- 
other image. The events detector 38 compares 
each of the digitized samples 74 and 80 of the 
current frame i with a corresponding digitized sam- 



ple 74 and 80 of a previous frame, for example, the 
frame i-k,. The counterpart samples are then com- 
pared to determine whether a scene change has 
occurred. More specifically, the detection of a 
5 scene change occurs as follows: 

1 . Generate, for example, 56 samples from a 
portion of each frame of the video envelope 72, for 
example, from the middle 60% of each frame; 

2. Compare counterpart samples 74 of the 
w current frame i and the previous frame i-k,; 

3. Calculate the absolute value of the dif- 
ference of each of the counterpart samples 74; 

4. Calculate the average of the absolute val- 
ues; and 

75 5. Indicate a scene change when the aver- 

age of absolute values exceeds a predetermined 
threshold value. Where the scene change occurs 
and k, equals 1, a sudden scene change is in- 
dicated, and when k, is greater than 1, a slow 

20 scene change is indicated. 

The station monitoring unit 10 monitors the 
broadcast signal and extracts signatures from the 
particular monitored signal when certain other pre- 
determined events are detected, which may, for 

25 example, by any one of the following: 

1. A predetermined time interval <e.g., 3-10 
seconds) since the previous predetermined event 
this predetermined time interval may be deter- 
mined, for example, by providing a time-out timer 

30 having the predetermined time period; 

2. A color change in successive frames or 
portions thereof; such color change may be deter- 
mined, for example, by detecting the average am- 
plitude of a particular color component, such as a 

35 blue component of each frame or a portion thereof 
and making comparisons with a previous frame of 
the type described hereinabove; and 

3. An audio level change, for example, by 
detecting the average magnitude of tfie audio sfg- 

40 nal for each frame, and making comparisons with a 
previous frame of the type described hereinabove. 

After the station monitoring unit 10 detects the 
occurrence of certain ones of the predetermined 
events, for example, the slow scene change, the 

46 blank frame and time-out events, the events detec- 
tor 38 detects a stability condition, and then pro- 
vides enable signals to the audio and video signa- 
ture extraction circuits 34, 42. The events detector 
38 compares the frame i wherein the predeter- 

50 mined event occurs with a subsequent frame, for 
example i + k„ The counterpart samples are com- 
pared to determine if a stability condition has oc- 
curred. More specifically, the detection of a stabil- 
ity condition occurs as follows: 

55 1. Generate corresponding samples, for ex- 

ample, 56 samples from a portion of a subsequent 
frame after the occurrence of the certain predeter- 
mined event; 
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2. Compare counterpart samples (74) of the 
frame i and a subsequent frame i + k 2 ; 

3. Calculate the absolute value of the dif- 
ference of each of the counterpart samples (74); 

4. Calculate the average of the absolute val- 
ues; 

5. Indicate a stability condition if the average 
of the absolute values is less than a predeter 
mined threshold value; and 

6. If a stability condition is not indicated, 
Increment i and repeat the above steps using the 
new value for i. 

After the station monitoring unit 10 detects the 
occurrence of the stability' condition following cer- 
tain detected predetermined events, the events de- 
tector 38 provides enable signals to the audio and 
video signature extraction circuits 34, 42. As de- 
scribed hereinbefore, the video signature and audio 
signatures are extracted, combined and stored 
along with the time of occurrence status and station 
Identification codes In the data storage device 44. 

Referring to FIG. 5, there is illustrated a flow 
chart of the logical steps performed by the station 
monitoring system according to the invention. The 
station monitoring unit 10 samples the broadcast 
signal to detect the predetermined events, such as 
the blank frame, sudden and slow scene changes. 
This is accomplished, for example, by taking 56 
samples from the middle 60% of each frame of the 
video envelope. The calculated average value D m 
of the present sampled frame I is compared to a 
predetermined threshold value T 1v and thereby a 
determination is made as to whether or not a blank 
frame has been broadcast When the average value 
D w is determined to be less than or equal to the 
threshold value T w , a decision is made that a blank 
frame has been broadcast Following the detection 
of a blank frame, an average value of Dqa of the 
audio energy signal 25 is calculated and compared 
to a predetermined threshold value T 1A . A deter- 
mination is made that silent audio is detected when 
the calculated average value Dqa is less than or 
equal to the threshold value T 1A . Following a deter- 
mination that silent audio has been detected, a 
blank frame code is stored In the data storage 
device 44. Next a time-out timer or counter for 
extracting signatures from the broadcast signal at 
the predetermined time Interval since the periodic 
predetermined event indicated by the character k, 
Is set to zero. A time-out timer or counter tor 
adjusting the stability condition determination, in- 
dicated by *e character kk, is set to zero. The 
present sampled frame I is reset and the average 
value D ov is calculated and is compared to a 
predetermined threshold value T, ( ^ to determine 
whether or not a non-blank frame has been broad- 
cast following the detected blank frame. When the 
average calculated value O w is greater than or 



equal to the predetermined threshold value T, ( *> , 
a determination is made that a non-blank frame has 
• been broadcast. Following such detected non-blank 
frame, a stability condition is detected prior to 
5 extracting a signature from the broadcast signal as 
is described hereinafter. When a determination is 
made that a non-blank frame has not been broad- 
cast the next frame i + 1 is sampled and the 
average value D w is calculated to continue until a 
w non-blank frame is detected. 

When a determination is made that a blank 
frame has not been broadcast, a determination is 
made as to whether or not a sudden scene change 
has occurred. A sudden scene change is detected 
75 by first calculating the average value of the dif- 
ference of each sample of a current frame i and an 
immediately prior frame i-1 , calculating the average 
of these values, shown in FIG. 5 as D 1S , and 
comparing the average of the absolute values D 18 
20 with a predetermined threshold value T^ When 
the average absolute values D 1S are determined to 
be greater than or equal to the threshold value Tzs. 
a decision is made that a sudden scene change 
has occurred. Following the detection of a. sudden 
26 scene change, a signature is extracted from the 
broadcast signal. The signature may be extracted, 
for example, by extracting a video signature, wait- 
ing a predetermined time period, T0, for example, 
1/3 seconds and extracting an audio signature. The 
x extracted video signature and the extracted audio 
signature are then combined into a composite sig- 
nature. The composite signature may be formed by 
concatenating the audio and video signatures into a 
single string of hexadecimal numbers. A date time 
35 code, status code and station Identification code 
are added to the composite extracted sign atur e 
which is stored In the data storage device 44. 

When a decision is made that a sudden scene 
change has not occurred, a determination is made 
40 as to whether or not a slow scene change has 
occurred. A stow scene change is detected by first 
calculating the absolute value of the difference of 
each sample of a current frame I and a previous 
frame i-k, (as illustrated in FIG. 4), calculating the 
45 average of these absolute values, shown equal to D 
1* and comparing the average of the absolute 
vetoes Dip with- a predetermined threshold value 
Tap. When the average absolute values D 1P are 
determined to be greater than or equal to the 
so threshold value Tap, a decision Is made that a stow 
scene change has occurred. When a decision is 
made that either a stow scene change has oc- 
curred or a non-blank frame is detected following a 
detected blank frame with silent audio, an enable 
as block is set shown as F-1, and sampling of the 
frame continues until a stability condition is de- 
tected. In addition, the time-out timer for forced 
triggering k enables the subsequent steps in the 
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signature extraction process after the elapse of a 
predetermined time period, shown as k* following a 
decision that F is not equal 1 and a slow scene 
change has not occurred. . 

Following the detection of certain predeter- 
mined events including a blank frame with silent 
audio followed by a non-blank frame, a slow scene 
change or time-out force triggering event, a stabil- 
ity con ditlon is detected prior to extracting a 
signature from the broadcast signal. A stability con- 
dition may be detected, for example, by comparing 
the frame i with a subsequent frame i + ki and 
determining when the magnitude of the difference 
D2 between the absolute values of the samples of 
the two frames is less than a predetermined thresh- 
old value T3. When the difference D2 is deter- 
mined to be less than or equal to the threshold 
value T3, a decision is made that a stability con- 
dition has been detected. Otherwise, the time-out 
timer kk is provided such that the comparing pro- 
cess is continued for a predetermined time period, 
shown as k«, for example 3-5 seconds. When a 
stability condition is not detected prior to the 
elapse of the time period k*. resetting of the thresh- 
old value T3 to a higher value, for example 50% 
higher, is enabled. In addition, where a stability 
condition is not detected before the lapse of the 
predetermined time period k*, a signature will not 
be extracted until a next predetermined event is 
detected. 

When a stability condition is detected, i.e., 
when the difference D2 in absolute values is less 
than or equal to T3, a signature is extracted from 
the broadcast signal. Following the storing of the 
composite extracted signature, the forced triggering 
time-out timer k, the stability condition time-out 
timer kk and the present frame counter i are reset 
to zero. 

Referring now to FIGS. 6 and 7, there are 
illustrated flow charts of the logical steps per- 
formed by the recognition and post-processing unit 
54 according to the invention. A two-level recogni- 
tion method is used to identify the particular repet- 
itively broadcast programs. 

FIG. 6 illustrates a first signature level recogni- 
tion and RG. 7 Illustrates a secondary program 
level recognition according to the method of the 
inven tion. Considering first the logical steps shown 
in RG. 6, the signature recognition computer 60 - 
(RG. 2) identifies and stores the blank frame code 
for use in the secondary program level recognition. 
The extracted signature Is compared with a plural- 
ity of reference signatures, for example, 40,000 
reference signatures, stored in the reference signa- 
ture data base 62 to determine a match between 
the extracted signature and a reference signature. 
A variable range hash code search of the stored 
reference signatures is performed to select possi- 



ble candidates therefrom. The absolute difference 
Dsiq between the extracted signature and the refer- 
ence signature candidate is calculated and com- 
pared to a predetermined threshold value Tao to 

s determine a possible match between the extracted 
signature and the reference signature candidate. 
When the absolute difference D S ig is greater than 
the predetermined threshold value T S iq the variable 
range hash code search is continued. 

70 When the absolute difference D S k3 is less than 
or equal to the predetermined threshold value Tga 
indicating a possible match, a correlation coeffi- 
cient is calculated between the extracted signature 
and the reference signature- candidate. The correla- 

is tion coefficient C Is compared to a predetermined 
low threshold value Cuj to determine whether or 
not the reference signature candidate is a possible 
match. When the correlation coefficient C is greater 
than or equal to the predetermined low threshold 

20 value Clq, a match is indicated and the possible 
matched reference signature is stored for use in 
the secondary problem level recognition. Otherwise 
in the recognition file 64, the variable range hash 
code search is continued. 

25 Following an indicated match determination, 
the correlation coefficient C is compared to a pre- 
determined high threshold value Chi to determine 
whether or not another reference signature can- 
didate should be selected. If the correlation coeffi- 

30 cient C is less than the threshold value Chi, the 
variable range hash code search is continued. 
When the correlation coefficient C is greater than 
or equal the high threshold caJue C w , another 
extracted signature is read from the reference sig- 

35 nature data base 62 and the first signature level 
recognition steps are repeated. 

The secondary program level recognition of 
RG. 7 includes reading the stored matched .or 
recognition signatures from the signature recogni- 

40 tion file 64, calculating the number of recognized 
signatures in a series with the identical program 
identifying name and determining the sequential 
order of the recognized signatures. Then a 
predefined decision rule is employed to identify or 

45 recognize a particular repetitively broadcast pro- 
gram. To recognize a particular broadcast program, 
the predefined decision rule may require a mini- 
mum number, for example two, recognized signa- 
tures that indicate an Identical program name and 

so have a predefined sequential order, and/or a major- 
ity of the number of recognized signatures read 
between stored blank frames to indicate an iden- 
tical program name. The particular recognized 
broadcast program thus Identified is stored and the 

55 post-processing computer 66 produces a file or log 
of the identified programs. 
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Obviously, modifications and variations of the 
present invention are possible in light of the above 
teachings. Thus, it is to be understood that, within 
the scope of the appended claims, the Invention 
may be practiced otherwise than as specifically 
described hereinabove. 



Claims 

1. A system for identifying a plurality of defined 
broadcast programs in a signal utilizing signature 
extraction, characterized by: 

means for detecting the occurrence of a predeter- 
mined event In the signal; 

means for detecting the occurrence of a stability 
condition in the signal after the occurrence of said 
predetermined events; and 

means for extracting the signature from the signal 
after the occurrence of said stability condition. 

2. The system as recited in claim 1 wherein 
the signal is a television broadcast signal and said 
predetermined event detecting means includes 
means for detecting a slow scene change from one 
video scene to another video scene in the televi- 
sion video signal. 

3. The system as recited in claim 1 wherein 
the signal Is a television broadcast signal and said 
predetermined event detecting means includes 
means for detecting a blank frame having silent 
audio followed by a non-blank frame In the televi- 
sion video signal. 

4. The system as recited in claim 1 wherein 
the signal Is a television broadcast signal and said 
predetermined event detecting means includes 
means for detecting a predetermined magnitude 
change in the television audio signal. 

5. The system as recited in claim 1 wherein 
the signal is a television broadcast signal and said 
predetermined event detecting means includes 
means for detecting a predetermined amplitude 
change in a selected color component of the televi- 
sion video signal. 

6. The system as recited in claim 1 wherein 
said predetermined event detecting means in- 
cludes means for detecting the elapse of a pre- 
determined time period after the occurrence of a 
previous predetermined event 

7. The system as recited in claim 6 wherein 
said predetermined time period Is approximately 3 
-10 seconds. 



8. The system as recited in claim 1 wherein 
the signal is a radio broadcast signal and said 
predetermined event detecting means includes 
means for detecting a predetermined change in 

5 magnitude of the radio signal. 

9. The system as recited in claim 1 wherein 
the signal is a television broadcast signal and said 
means for detecting the occurrence of said pre- 
determined event includes means for low pass 

w filtering the video signal to produce a video en- 
velope signal, means for periodically sampling said 
video envelope signal to provide a series of video 
samples, and means for comparing said video 
samples and a predetermined threshold value to 

rs indicate a blank frame wherein said video samples 
are less than said predetermined threshold value. 

10. The system as recited in claim 9 wherein 
said means for low pass filtering the video signal 
has a frequency response of approximately 0 to 2 

ao kHz. 

11. The system as recited in claim 9 wherein 
the means for sampling said video envelope signal 
includes means for taking a predetermined number 
of samples of said video envelope signal and 

as means for taking the average of predetermined 
consecutive samples to generate a plurality of 
average signals, each average signal defining one 
of the features that comprises a feature string. 

12. The system as recited in claim 1 wherein 
30 the signal is a television broadcast signal and said 

means for detecting the occurrence of said pre- 
determined event includes means for low pass 
filtering the video signal to provide a video en- 
velope signal, means for periodically sampling said 

ss video envelope signal to provide a series of video 
samples and means for comparing said series of 
video samples to a previous series of video sam- 
ples to indicate a stow scene change predeter- 
mined event wherein the absolute value of the 

<o difference of said series of samples is greater than 
a predetermined threshold value. 

13. The system as recited in claim 12 wherein 
said means for low pass filtering said video signal 
has a frequency response of approximately 0 to 2 

46 kHz. 

14. The system as recited in claim 1 wherein 
said means for detecting the occurrence of said 
pre-determined event includes means for process- 
ing said audio spectrum signal to provide a plural- 

« ity of audio spectrum signals, and means for 
digitizing said audio spectrum signals, and compar- 
ing said digitized audio spectrum signals to pre- 
viously produced digitized audio spectrum signals 
to indicate a predetermined event wherein a pre- 
ss determined change in magnitude is detected. 

15. The system as recited in claim 14 wherein 
the step of processing said audio signal is provided 
In the range of approximately 330 Hz to 2.54 kHz. 
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16. The system as recited in claim 1 wherein 
said means for extracting a signature from the 
broadcast to be identified includes means for low 
pass filtering a video signal to provide a video 
envelope signal, means for periodically sampling 
said video envelope signal to provide a series of 
video samples, means for digitizing said video 
samples to provide a video signature, means for 
processing an audio signal to provide a plurality of 
audio spectrum signals, means for digitizing said 
audio spectrum signals to provide an audio signa- 
ture, and means for combining said video signature 
and said video signature. 

17. The system as recited in claim 16 wherein 
said means for combining said video signature and 
said audio signature includes means for defining a 
first feature string representative of said video sig- 
nature, means for defining a second feature string 
representative of said audio signature, and means 
for concatenating said first and second feature 
strings. 

18. The system as recited in claim 1 wherein 
said means for detecting the occurrence of a 
stabiliity condition includes; 



means for low pass filtering a video signal to pro- 
vide a video envelope signai; 

means for periodically sampling said video en- 
5 velope signal to provide a series of samples; 

means for calculating an absolute value of the 
difference of said series of video samples and a 
previous series of video samples; and 

w 

means for comparing said calculated absolute val- 
ues and a predetermined threshold value to in- 
dicate the stability condition wherein said calcu- 
lated absolute values are less than said predeter- 

75 mined threshold value. 

19. The system as recited in claim 18 further 
including means for increasing said predetermined 
threshold value wherein a predetermined time pe- 
riod elapses after the occurrence of said detected 

20 predetermined event without detecting a stability 
condition. 
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